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Since observers' memory-based ratings of 
organizational phenomena provide data in research and decision-making 
contexts, bias in observers' judgments must be examined. A study was 
conducted to explore the extent to which leader behavior ratings are 
more generally biased by the availability heuristic. The availability 
heuristic isoperative when a rater judges the frequency of 
occurrence of a behavior by the ease with which specific instances of 
that behavior can be -remembered which certain 

character istics of behavior description items underly differences in 
the way examples of these behaviors are retrieved from memory was 
investigated. A literature review identified six possible 
determinants of availability differences between behavior description 
items. College students (N=96) rated each behavior description of the 
Leader Behavior Description_Questionnaire (tBDQ) onone of the six 
identified availability components. All 100 items of the LBDQ were 
rated separately by 16 subjects_for each availability component. The 
results strongly support the hypotheses that availability of items 
(i.e., more specific^ valerit, imagineabie , dramatic, familiar and 
retrievable) would significantly correlate with actual ratings of 
leader behavior , reported in two previously published studies , and 
that these correlations would be larger when little actual stimulus 
behavior was present prior to rating compared to conditions where 
actual behavior was present. (Author/NB) 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
************************************* 



ERLC 



Heuristic Processes in Ratings of Leader Behavior: Assessing 
Item-induced Availability Biases 

John F. Binning and Guadalupe Fernandez 
Illinois State University 



Paper presented at the 94th Annual Convention of the American 
Psychological Association, Washington, D. C, August, 1 986. 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EOUCA/foNAL RESOURCES INFORMATION 
_ / CENTER (ERIC) 

Q^Dii document has been re^bduced as 
''received from the person or organization 
P/iQinating it 

n .MlhOX chanQes have been made to improve 
reproduction quality. 



Points of view or opinions stated in this docu- 
ment a*0- not- necessarily represent official 
OERI position or policy. 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



Heuristic Processes in Ratings of Leader Behavior: Assessing 
Item-induced Availability Biases 

The present study investigated the extent to which certain charac- 
teristics of behavior description items underly differences in the way 
examples of these behaviors are retrieved from memory- A review of the 
literature resulted in the identification of six possible determinants of 
availability differences between behavior description items. All 100 items 
of the Leader Behavior Description Questionnaire (LBDQ) were assessed on 
each of the six availability copmponents. It was hypothesized that the 
availabil ity of items (i.e., more specific, valent, imagiheable, dramatic, 
familiar and retrievable) would sighificatly correlate with actual ratings 
of leader behavior, reported in two previously published studies. 
Moreover, it was hypothesized that these correlations would be larger when 
little actual stimulus behavior was present prior to rating compared to 
conditions where actual behavior was present. These hypotheses were 
strongly supported. 
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Observer's memory-based ratings of various organizational phenomena 
provide the data in many research and decision-making contexts (e.g., stu- 
dies of leader behavior, performance appraisal, selection interviewing, 
etc.). Therefore, any bias in observers' judgments poses a threat to the 
quality of these data as well as subsequent decisions. She such bias 
occurs when group members rate various aspects of their group's functioning 
with concomitant knowledge of the group's performance (Binning & Lord, 
1980; Larson, 1982; Lord, Binning, Rush & Thomas, 1978; Staw, 1975). It is 
consistently found that group members given bogus feedback indicating their 
group performed a designated task poorly, rate the group leader's behavior 
(as well as other group phenomena) significantly more unfavorably than 
their counterparts who receive bogus good performance feedback. Binning, 
Zaba & Whattam (in press) suggest that this performance cue effect stems 
from use of the representativeness heuristic (Tversky & Kahneman, 1982). 
This simple judgmental strategy is used to simplify judgments about the 
likelihood that an event (or behavior) is an instance of a given class of 
events based on how representative the event is of the more general class. 
Binning, et al. (in press) found that behaviors more closely associated 
with preheld notions of effective leadership were judged to have more 
likely occured in the bogus good performance condition and vice versa for 
the poor performance condition, independently of the leader's actual beha- 
vior! 

The present study explores the extent to which leader behavior ratinqs 
are more generally biased by another cognitive heuristic commonly relied 
upon for making judgments of frequency and probability. The availability 
heuristic is operative when a rater judges the frequency of occurence of 
some behavior or event by the ease with which specific instances of that 
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behavior can be brought to mind, A behavior's availability in memory is 
affected not only by actual frequency of exposure to that behavior, but 
also other factors unrelated to actual frequency of exposure. These other 
factors operate to bias judgments based on availability. 

The logic underlying the present study suggests that behavioral 
description items, as typically found on leadership questionnaires, job 
analysis questionnaires, and performance appraisal instruments, for 
instance, may differ in semantic characteristics, level of description, arid 
potentially many other aspects which make some more "available" than 
others. It is important to stress that the locus of these differences is 
in the ease with which examples of the Behaviors described can be recalled 
from memory, independently of actually observed behavior. Those behaviors 
for which it is easier to imagine examples, for instance, would be judged 
as more frequent (or more likely to occur) than less available behaviors, 
perhaps even if the opposite were true of actual stimulus behavior. This 
could potentially represent a potent source of bias in ratings of observed 
behaviors . 

To test this notion, the literature was first reviewed in order to 
identify various faetors which might under ly availability differences among 
behavior description items. A total of six possible determinants of 
availability differences was incientif ied. These were (a) Dramatic Value 
(b) Familiarity (c) Imagery Value (d) Specificity (ej Valence arid 
(f) Retrieval: ility. Stated simply, we hypothesized that behavioral items 
which described behaviors which (a) aroused more emotion (drama) (b) were 
more familiar to subjects (c) aroused more vivid mental images (d) were 
more specific (e) seen as more evaluatively positive (valence) and (f) 
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were independently judged to easier in retrieving examples from memory* 
would be rated to occur with greater frequency than their less available 
counterparts. 

Method 

Subjects and Procedure 

Ninety-six student subjects were given the Leader Behavior Description 
Questionnaire (LBDQ, Form Ml, Stogdill, 1973), with special instructions to rate 
each behavior description on one of the six availability components 
described above. In other words, all 199 items on the LBDQ were rated 
separately by sixteen subjects for each availability component. 

Instructions for each of the components contained a brief definition 
of the component followed by a simple example. For instance, those sub- 
jects instructed to rate dramatic value were told; 

Behaviors differ in the extent to which they evoke strong feelings in 
us. Some behaviors evoke strong feelings in us; for example, "my father 
just won a million dollar lottery. 11 Qther behaviors do not evoke strong 
feelings; for example, "John watched TV for a couple of hours." 

A similar format for instructions was used for the other five components. 
Subjects rated each LBDQ item on a 19-point Likert scaled anchored at the 
ends with descriptions such as, "very easy to imagine - very difficult to 
imagine" or "do not evoke strong feelings-evoke strong feelings% etc, The 
mean rating for each LBDQ item on each availability component was then com- 
puted. As a result, each of the 199 LBDQ items had associated with it, six 
values representing its mean rating for each of the six availability com- 
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ponents. 

In order to test our hypothesis, we needed actual ratings of leader 
behavior obtained from an independent sample of raters. Therefore, the 
leader behavior ratings from two previously published studies were used. 
These two studies were chosen because of the important contrast in methodo- 
logy they afforded. Rush, Thomas and Lord (1977), interested in assessing 
implicit leadership theories, haa their subjects complete the LBDQ after 
observing no actual leader behavior. Instead, subjects were given a brief 
description manipulating several characteristics of a hypothetical leader. 
This limited information condition is in stark contrast to a study by Lord, 
et al. (1S78) where subjects were exposed to a fifteen minute videotape of 
actual leader behavior prior to completing the LBDQ. We refer to this 
latter study as a full information condition. 

We then correlated, across LBDQ items, their availability ratings with 
the actual leadership ratings obtained in the limited and full conditions. 
It had been hypothesized that aetual ratings of leadership would positively 
covary with availability components in the manner described earlier. We 
further hypothesized that the magnitude of these correlations would differ 
between the limited and full information conditions. In the limited infor- 
mation condition, where actual leader behavior information is minimi zed* 
reliance on heuristically-generated judgments should be greater. On the 
other hand, in the full information condition, where actual rating-relevant 
information is presented, leader behavior ratings should be lsss dependent 
on heuristical ly-based judgments. Therefore, we hypothesized that for both 
the limited and full information conditions, significant correlations bet- 
ween availability values and actual ratings would be obtained. However, we 



expected that these correlations would be substantially larger in the 
limited than in the full information condition. 

Results 

In order to assess the independence of the six availability com- 
ponents, their ratings were intercorrelated. This intarcorrelation matrix 
appears in Table 1. The components are moderately intercorrelated, the 



Insert Table 1 about here 

largest being the correlation between familiarity and valence (r = .73, 
df =98* p < .001). Familiarity also correlates highly with retrievabil ity 
(r = .72, p K ,001). The remaining significant intercorrelations range 
in absolute value from .18 (p < .05) to .63 (p < .001). The median corre- 
lation is .44. Three of the intercorrelations were not statistically 
significant. Generally then, the six components, while not being indepen- 
dent, seem to be tapping somewhat different character! sties of the behavior 
description items. 

Correlations between availability components and actual ratings of 
leadership were then computed. These appear in Table 2. All six avail abi- 



Insert Table 2 about here 

lity components correlated significantly with actual ratings in both the 
limited and the full information conditions, with the exception of specifi- 
city and imagihabil ity, which did not correlate in the full information 
condition. The correlations in the limited information condition ranged 
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from .27 to .74, and in every case, as expected, were larger than the 
corresponding correlations in the full information condition. 

To more fully assess the role of availability in determining actual 
ratings, step-wise regression analyses were performed. In the limited 
information condition, familiarity, retrievabil ity and drama entered the 
equation explaining* 54%, 4% and 4% of the variance in actual ratings, 
respectively. In the full information condition, valence and retrievabi- 
1 ity explained 19% and 4%, respectively. 

discussion 

The results of this study strongly suggest that ratings of leader 
behavior, and presumably other behaviors, are significantly biased by the 
ease with which examples of the behavior descriptions (or anchors) can be 
retrieved from memory. Although this bias is greater when there is limited 
information about actual behavior, significant bias exists even when clear 
stimulus Behavior has been observed. When used to rate actually observed 
behavior, the more familiar, dramatic, specific, positively valent, imagi- 
neable and retrievable behavior description items, received higher fre- 
quency ratings, than less available items. Moreover, although the six 
availability components were intercorrelated, regression analyses indicate 
that several do explain unique variance in actual ratings. The upshot of 
these findings is that behavior ratings may be susceptible to item-induced 
biases which result from raters 1 naturally-occuring use of the availability 
heuristic as an aid in processing behavioral information. By better 
understanding the heuristic processes which underly behavior ratinqs, prac- 
ti oners and researchers alike, may be better able to eliminate such 
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unwanted bias from widely-used rating instruments. 

It is important to emphasize that, unTike rating biases which derive 
primarily from the interaction between cognitive processes and stimulus 
information (i.e., performance cue effects mentioned earlier), the availa- 
bility bias described here may stem primarily from identifiable differences 
between behavior description items. This being the case, rating instru- 
ments can be more carefully developed so that items and anchors are equated 
for availability in the construction phase of development. Leader behavior 
questionnaires along with BARS, BOS and other standard rating formats can 
be constructed to ameliorate such biases. 
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Table 1 



Intercorrelations Among Availability Components 



F D S V R 

Component 

Familiarity 

Drama (D) .57*** 

Specificity (S) .33*** .15 

Valence (V) .73*** .63*** .31*** 

Retrievabil ity (R) .72*** .43*** .33*** .55*** 

Imaginability (I) .27** .11 .18* .00 .45*** 



Note: df = 98 

*** p < .501 

** p ? .01 

* p < .05 
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Table 2 

Correlations of Availability Components with Actual Behavior Ratings 



Components 


Limited 
Information 
Condition 


Full 

Information 
Condition 


Fanri 1 iarity 


. 74*** 


. 40*** 


Drama 


.27** 


.19* 


Specificity 


.35*** 


.16 


Valence 


.47*** 


. 44*** 


Retrievability 


.68*** 


.41*** 


Imaginability 


.38*** 


.04 



*** p < .001 
** p i < .01 
* p < .05 
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